Skip to content

Use unified API for prompt voice agents - #9634

Open
JianW (v1212) wants to merge 17 commits into
Azure:mainfrom
v1212:users/wujia/prompt-voice-agent-unified-api
Open

Use unified API for prompt voice agents#9634
JianW (v1212) wants to merge 17 commits into
Azure:mainfrom
v1212:users/wujia/prompt-voice-agent-unified-api

Conversation

@v1212

@v1212 JianW (v1212) commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Switches prompt voice agent deploys to the unified /agents API with the flat audio.output.voice payload shape.
  • Keeps the user flow unchanged: prompt voice remains behind AZD_AI_AGENT_ENABLE_PROMPT_VOICE, and users do not need to configure an API mode.
  • Adds unified create/update behavior, persists AGENT_<SERVICE>_VERSION, and stores callable wss://.../endpoint/protocols/voice?api-version=v1 endpoints.

Why

  • Regional API matrix testing showed no region where the old prompt voice endpoint worked while the unified /agents flat shape failed.
  • In regions where prompt voice managed mode is available, the unified /agents flat shape worked.
  • Remaining failures were due to preview enablement or managed model availability, not observed API-shape incompatibility.
  • This gives Support advanced prompt voice agent settings #9655 a stable unified API base for advanced prompt voice settings.

Validation

  • go test ./... -count=1
  • go build ./...
  • cspell lint 'extensions/**/*.go' 'extensions/**/*.md' --config ./.vscode/cspell.yaml --no-progress
  • Regional API matrix for prompt voice managed create across supported hosted-agent regions.
  • Private test guide validation in West US 2: managed, BYOM realtime, and BYOM cascaded agents deployed successfully; /agents/{name} GET returned flat output shape and non-empty versions; sample UI server startup was verified.

Fixes #9654

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
20 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions github-actions Bot added the ext-agents azure.ai.agents extension label Aug 19, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
20 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds opt-in unified API deployment modes for prompt voice agents while retaining legacy behavior.

Changes:

  • Adds legacy, unified, and unified-flat deployment modes.
  • Supports unified create/update, version persistence, and WebSocket endpoints.
  • Adds flat voice payload models, documentation, and tests.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
README.md Documents voice API modes and behavior.
service_target_agent.go Implements mode selection, deployment, and persistence.
service_target_agent_test.go Tests mode resolution and endpoints.
agent_yaml/map.go Builds flat voice payloads.
agent_yaml/map_voice_test.go Tests flat voice mapping.
agent_api/operations.go Adds unified API operations.
agent_api/operations_test.go Tests unified routes and headers.
agent_api/models.go Defines flat voice models.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go Outdated
Comment thread cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go Outdated
Copilot AI review requested due to automatic review settings August 20, 2026 04:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map_voice_test.go:250

  • [azd-code-reviewer] These unchecked assertions can panic when the generated wire shape changes, hiding which field is malformed. Guard each decoded JSON level so the test fails with a localized diagnostic instead.
	definition := wire["definition"].(map[string]any)
	audio := definition["audio"].(map[string]any)
	output := audio["output"].(map[string]any)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:2334

  • [azd-code-reviewer] The new create/update decision tree is not covered by the added tests. The client tests verify each HTTP method in isolation, but they would not catch regressions where a 404 updates, a 200 creates, or a non-404 GET error is ignored. Add orchestration tests for successful GET, 404, and another error status, asserting which follow-up request is issued.
	remoteAgent, getErr := agentClient.GetVoiceAgentUnified(
		ctx, request.Name, agent_api.AgentEndpointAPIVersion, overriddenHost,
	)
	if getErr == nil && remoteAgent != nil {
		progress("Updating voice agent using unified API")

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:2186

  • [azd-code-reviewer] This new os.Getenv input is missing from the canonical environment-variable reference. cli/azd/docs/environment-variables.md is the source of truth and already has an azure.ai.agents section; add AZURE_VOICE_AGENT_API there with its accepted values, precedence, and legacy default so the README and reference do not drift.
		mode = strings.TrimSpace(os.Getenv(voiceAgentAPIEnvKey))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the timeline of this in the backend? I really don't like having all of this backcompat support for something that's still private preview and clearly in flux. If the backend is actively rolling out, then we should wait for that to complete and just make the full switch.

Comment thread cli/azd/extensions/azure.ai.agents/README.md Outdated
…ice-agent-unified-api

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/README.md
Copilot AI review requested due to automatic review settings August 21, 2026 01:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (2)

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map_voice_test.go:250

  • azd-code-reviewer: These unchecked assertions turn an unexpected serialized shape into a package-stopping panic instead of a useful test failure. Guard each lookup and type assertion before descending into the next map.
	definition := wire["definition"].(map[string]any)
	audio := definition["audio"].(map[string]any)
	output := audio["output"].(map[string]any)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:2334

  • azd-code-reviewer: The core 200-vs-404 dispatch is not exercised by automated tests. The added client tests verify each HTTP method independently, so a regression that selects update after a 404, creates after a 200, or mutates after another GET failure would still pass. Add table-driven coverage around this branch for 200, 404, and non-404 failures.
	remoteAgent, getErr := agentClient.GetVoiceAgentUnified(
		ctx, request.Name, agent_api.AgentEndpointAPIVersion, overriddenHost,
	)
	if getErr == nil && remoteAgent != nil {
		progress("Updating voice agent using unified API")

Copilot AI review requested due to automatic review settings August 24, 2026 11:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (1)

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map_voice_test.go:250

  • azd-code-reviewer: Guard these JSON map assertions so a malformed wire shape reports the failing level instead of panicking before the field assertions run.
	definition := wire["definition"].(map[string]any)
	audio := definition["audio"].(map[string]any)
	output := audio["output"].(map[string]any)

Comment thread cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/models.go Outdated
Comment thread cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go Outdated
Copilot AI review requested due to automatic review settings August 25, 2026 01:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings August 25, 2026 01:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:2220

  • azd-code-reviewer: ENDPOINT is not a reliable final-write marker on redeploy because an existing value is never cleared. If the agent name changes and persisting NAME or VERSION fails, both isDeployed and Endpoints still accept the stale endpoint and direct users to the previous agent despite the failed deployment. Clear the marker after the remote operation succeeds, then write the new endpoint last.
	for _, envVar := range []struct{ key, value string }{
		{fmt.Sprintf("AGENT_%s_NAME", serviceKey), agentObject.Name},
		{versionKey, versionValue},
		{fmt.Sprintf("AGENT_%s_ENDPOINT", serviceKey), baseEndpoint},

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state_test.go Outdated
Copilot AI review requested due to automatic review settings August 25, 2026 01:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go:604

  • azd-code-reviewer: Script-bearing Azure locales are serialized incorrectly. For example, the valid Azure voice az-Latn-AZ-BanuNeural is classified as OpenAI by azureNeuralVoicePattern, while this split would derive only az-Latn rather than az-Latn-AZ. The unified payload then omits or corrupts voice_locale and lowercases the voice name. Update both the Azure classifier and locale extraction to support BCP-47 script subtags, and add a script-locale test.
func voiceWireLocale(voice *agent_api.VoiceConfig) string {
	if voice == nil || voice.Name == "" || isOpenAIVoice(voice.Name) {
		return ""
	}
	parts := strings.SplitN(voice.Name, "-", 3)
	if len(parts) < 2 {
		return ""
	}
	return parts[0] + "-" + parts[1]

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:2279

  • azd-code-reviewer: A non-404 failure from the preliminary GET is returned as OpCreateAgent, although no create request was attempted. ServiceFromAzure embeds this operation in both the user-visible message and telemetry code, so an outage or authorization failure is misleadingly reported as create_agent. Add an OpGetAgent operation and return it for this branch.
	shouldUpdate, decisionErr := shouldUpdateVoiceAgent(remoteAgent, getErr)
	if decisionErr != nil {
		return nil, exterrors.OpCreateAgent, decisionErr

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:2228

  • azd-code-reviewer: This versioned voice deployment does not publish AGENT_<SERVICE>_PROJECT_ENDPOINT. validateFoundryAgentDependency requires that scope marker; its legacy fallback compares the new wss:// endpoint with the https:// project endpoint and therefore always fails. Any agent that declares this prompt-voice service in uses is still reported as not ready after a successful deploy. Persist the project endpoint before the final ENDPOINT marker, matching hosted-agent registration at service_target_agent.go:3295.
	for _, envVar := range []struct{ key, value string }{
		{fmt.Sprintf("AGENT_%s_NAME", serviceKey), agentObject.Name},
		{versionKey, versionValue},
		{endpointKey, baseEndpoint},

Copilot AI review requested due to automatic review settings August 25, 2026 01:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comment thread cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go Outdated
@v1212

Copy link
Copy Markdown
Contributor Author

Travis Angevine (@trangevi) Thanks for the latest review. I addressed the complete-pivot comments in the current revision:

  • Removed the prompt voice public README section; private-preview guidance stays in the private test guide.
  • Removed the legacy /voice_agents deploy path and AZURE_VOICE_AGENT_API mode switch.
  • Replaced the existing voice API models with the new /agents flat wire shape instead of adding Flat-suffixed models.
  • Replaced the Unified-suffixed client methods; GetVoiceAgent/CreateVoiceAgent/UpdateVoiceAgent now use /agents directly.
  • Cleaned up migration wording and stale comments.
  • Also handled the follow-up robustness items: clear ENDPOINT before final marker write, persist PROJECT_ENDPOINT, use OpGetAgent for preliminary GET failures, support Azure voice script locales, and guard JSON wire-shape assertions in tests.

Local validation passed: go test ./... -count=1, go build ./..., and cspell lint. CI is rerunning after the latest push.

Copilot AI review requested due to automatic review settings August 25, 2026 02:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state_test.go Outdated
Copilot AI review requested due to automatic review settings August 25, 2026 02:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Suppressed comments (1)

cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go:558

  • azd-code-reviewer: This does not fully enforce ENDPOINT as the voice completion marker. A unified voice deploy writes NAME and VERSION before ENDPOINT; if the final write fails, the voice branch below is skipped, NAME/VERSION pass the hosted guard, and stale protocol endpoints from an earlier hosted configuration can be returned. Preserve the isVoice result and return CodeMissingAgentEnvVars whenever its base ENDPOINT is empty instead of falling through to hosted endpoint collection.
	// Voice agents (kind: prompt-voice) use the base ENDPOINT as their callable
	// endpoint and deploy completion marker, and unified deploys also record
	// VERSION. Gate the base-endpoint path on the service's actual declared
	// kind (resolved via the shared agentkind lookup, so this agrees with the
	// deploy path and next-step reader) rather than on the env-var shape: a hosted

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext-agents azure.ai.agents extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Track prompt voice unified API deployment

4 participants